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Abstract 


Artificial intelligence (AI) is increasingly permeating our daily lives, and the field of education is 
no exception. Technology already plays a significant role in education, and AI is rapidly advancing. 
Chatbots, for instance, have been used as a valuable tool in schools for decades. With the emergence 
of tools like ChatGPT, their usage has expanded even further. The presence of such tools can be 
highly beneficial for teachers in the educational setting. The study focused on the fact that ChatGPT 
can serve as an excellent support for teachers in lesson planning. The usefulness of the tool and 
the challenges that teachers may encounter when using it to create lesson plans were explored. 
The results of the study, based on the analysis of 58 lesson plans created using ChatGPT, revealed 
certain limitations. Therefore, it is crucial to empower teachers to make prudent use of this tool. 
Keywords: artificial intelligence, chatbots in education, learning preparation, technology and 
engineering, STEM, natural science 


Introduction 


Artificial Intelligence (AI) is increasingly being integrated into our everyday lives 
in various ways and forms. The way individuals live in society is changing because of AI. 
Al-powered new technologies have been used in a variety of sectors, and the educational 
setting is no exception (Durso & Arruda, 2022). One of the most well-known forms of AI 
that is becoming a part of our daily lives is intelligent agents, which are created through 
the development and analysis of intelligent software and hardware. Their advantage lies 
in their ability to perform various tasks in a very short period of time (Adamopoulou & 
Moussiades, 2020). However, the ChatGPT tool, released in late November 2022, is not 
the first tool of its kind. The first chatbot, named ELIZA, was developed by Weizenbaum 
in 1966, but it was not widely known to the general public (Adamopoulou & Moussiades, 
2020; BradeSko & Mladenic¢, 2012). Chatbot assistants, such as Amazon's Alexa, Apple's 
Siri, and Google Assistant, have gained somewhat more recognition, allowing users to 
freely communicate with them in various languages. Despite the existence of such tools 
earlier, ChatGPT has gained tremendous popularity worldwide. ChatGPT is a generative 
AI tool that is capable of solving and executing highly complex tasks within a short span 
of time (Baidoo-Anu & Owusu, 2023). It represents the most advanced language model 
designed to generate text that is exceedingly challenging, and in some cases practically 
indistinguishable from text written by a human (Halaweh, 2023; Pericles 'asher' Rospigliosi, 
2023; Rudolph et al., 2023). 

Over the past few decades, both learning environments and teaching strategies 
have undergone steady change (Kerneza & Kordigel AberSek, 2022). The ChatGPT 
tool has gained immense popularity among users, including the area of education (and 
teachers), due to its usefulness in performing complex tasks directly or indirectly related 
to education. In their research, Clarizia et al. (2018) highlighted that chatbots can be a 
highly useful technology that supports education, particularly by significantly improving 
personalized learning and reducing the administrative workload of teachers. The potential 
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usefulness of such tools in education has also been highlighted by Cunningham-Nelson 
et al. (2019). They stated that chatbots represent a promising area for potential use in 
education as they can streamline and personalize educational components. Additionally, 
chatbots prove to be valuable in various areas of education, such as teaching and learning, 
administration, assessment, as well as research and development. Utilizing the ChatGPT 
tool enables users to enhance their writing skills, summarize information effectively, 
and format their content, leading to substantial time savings in their work. One notable 
advantage of ChatGPT is its ability to detect grammatical and stylistic errors, thereby 
providing valuable assistance and education to students and users of the tool in general 
(Atlas, 2023a; Atlas, 2023b). The utility of the ChatGPT tool extends beyond education 
and finds application in various other fields. In the realm of research, it proves beneficial 
to students by offering a wealth of information related to their chosen topics, thereby 
aiding in the development of their research skills. Moreover, the tool has the capacity to 
propose unexplored aspects, fostering students' curiosity and prompting them to delve into 
new research areas (Kasneci et al., 2023). Despite the potential positive aspects of using 
chatbots in education, it is important to acknowledge that there will be some scepticism. 
As demonstrated by the study conducted by Baidoo-Anu and Owusu (2023), the ChatGPT 
tool has evoked mixed feelings among teachers. The main concern lies in the fact that this 
AlJ-based tool brings about significant changes to existing educational practices. While the 
ChatGPT tool is relatively new, several studies have addressed the issue of text plagiarism 
when utilizing it. Khalil and Er (2023) conducted a study revealing that 80 % of essays 
written using the ChatGPT tool had a similarity score of 20 % or less compared to other 
texts. Susnjak (2022) also explored the capabilities of the ChatGPT tool and found that it 
exhibits a certain degree of critical thinking. Indeed, considering these developments, it is 
reasonable to question where such technological advancements will lead us in the future. 
It is crucial to contemplate how we will adapt to new tools that leverage AI. Furthermore, 
it is of utmost importance to harness the potential offered by such tools and utilize them to 
our advantage. There are undoubtedly numerous possibilities for teachers to incorporate 
tools like ChatGPT into their work in a meaningful manner. In our research, we primarily 
focused on exploring how teachers could effectively utilize the ChatGPT tool for preparing 
lesson plans. 


Research Problem 


The emergence of new tools that are becoming part of our daily lives presents 
opportunities and advantages in their usage. However, it is important to be aware of the 
potential pitfalls that we may encounter while using them. Nevertheless, these tools can 
be versatile and provide excellent support to teachers in their work, particularly in lesson 
planning and preparation. The use of new tools like ChatGPT also requires certain skills 
from the users. Most of the existing research primarily focuses on the opportunities that 
Al-supported tools like ChatGPT offer to teachers. However, only a few studies highlight 
the specific skills that teachers would need to utilize such tools effectively and efficiently 
in their work and derive support from them. The same holds true for the area of lesson 
planning. It is crucial to identify the skills that teachers require to become competent and 
discerning users of chatbots for teaching purposes and related administrative tasks. 
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Research Focus 


The aim of the research was to explore the feasibility of using the ChatGPT tool 
in the creation of an appropriate lesson plan in the areas of technology, science, and 
nature (STEM), while also exploring the basic skills required to create lesson plans in 
these subject areas. However, the objective was not to develop a complete lesson plan but 
rather to identify and outline the necessary steps that could serve as a solid framework for 
teachers to subsequently construct an appropriate lesson plan. Therefore, the hypothesis 
was formulated as follows: “With the assistance of the ChatGPT tool, it is feasible to 
generate suitable starting points that can serve as a solid foundation for creating an 
appropriate lesson plan”. 


Research Methodology 
General Background 


Since its release in November 2022, the ChatGPT tool quickly gained immense 
popularity among the broader educational community, as well as among students from 
various fields. Even students studying in pedagogical programs (pre-service teachers) who 
will become teachers upon completion of their studies were not an exception. In practice, 
it has been observed that students utilize this tool; however, they often struggle to critically 
assess and evaluate the content generated using the ChatGPT tool. In order to examine and 
explore whether the ChatGPT tool is suitable for designing educational lessons in the fields 
of technology, science, and nature, a study was conducted in March 2023. Pre-service 
teachers were tasked with using the ChatGPT tool to attempt to write a pedagogically 
grounded lesson plan for teaching the subject of Science and Technology (i.e., the realm 
of technology, science, engineering, and natural sciences - STEM). The data collection 
or preparation of lesson plans was conducted in a way that pre-service teachers had two 
different subject areas available, and they had to achieve three different learning objectives. 


Table 1 
Contents That the Students Considered When Creating a Lesson Plan Using ChatGPT 


Class 4th grade 


Learning unit Substance flow Changing the properties of substances 


to use basic processing to make useful items 
techniques for paper from different types of 
and wood materials, paper materials by using 
synthetic substances, various methods of 
thin sheet metal joining materials 


using wooden materials 
in making models and 
mock-ups; reading a 
plan and realizing it 


Learning 
objective 


The pre-service teachers were given the freedom to choose the field, unit, and 
learning objective based on their own interests. However, they were required to consider 
the condition that they had already written a lesson plan in the same subject area in the 
past. This requirement ensured that their work in this research study was based on the 
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knowledge and experience they had gained during the preparation of a traditional lesson 
plan. Their task was to ask the first question or command to the ChatGPT tool, which 
was “Write a lesson plan for educational work in 4th grade for the objective ” 
(insert the learning objective chosen by the pre-service teachers). Once the ChatGPT 
tool generated the first response and thus the basic lesson plan, pre-service teachers were 
allowed to ask further questions and write desired commands to the tool until they felt that 
the starting points of the lesson plan were appropriate. When they felt that the lesson plan 
was appropriate, they took a screenshot and uploaded it to a shared through the 1ka.si web 
application, which allows for anonymous submission. 


Sample 


The research was conducted in March 2022. The study employed a purposive non- 
random sampling method, involving 58 third-year students (pre-service teachers) from 
the Faculty of Education in Slovenia (Elementary Education program). The pre-service 
teachers' ages ranged from 20 to 22 years, with 87.94% female participants and 12.06 % male 
participants. The pre-service teachers’ group was selected based on their prior experience 
using ChatGPT, as they had already utilized the tool in previous academic assignments. 
Additionally, the selected pre-service teachers had previous teaching experience, which 
means they had practical classroom exposure and, consequently, practical experience 
in using lesson plans as teachers. This implies that they were familiar with the process 
of creating lesson plans from their previous studies, including the elements and content 
required. This prior knowledge was significant, as it enabled the pre-service teachers to 
understand the necessary information for a lesson plan and focus on how to obtain that 
information using the ChatGPT tool. The pre-service teachers participated in the study 
voluntarily, and they were provided with information regarding the research procedures 
and objectives. 


Instrument and Procedures 


Before conducting the main study, a pilot study was carried out, involving 9 
participants consisting of 3 students, 3 teachers, and 3 university professors. The purpose 
of the pilot study was for the participants to create lesson plans in the areas of literature, 
science, and technology using the ChatGPT tool. Based on their work, fundamental skills 
required for successful lesson plan creation using the ChatGPT tool in the fields of science, 
technology, and nature were analysed and identified. These skills were as follows: 

1. The ability to recognize and troubleshoot issues: it should be noted that 
the chatbot occasionally offers incorrect answers since the question is 
not comprehensible to it. The problem must be acknowledged by the 
individual in this role, and workable solutions need to be devised. 

2. Abstract thinking: the ability to think in an abstract manner, to recognize 
distinct content-related concepts, to link ideas, to uncover patterns, and to 
engage in analytical thought processes. 

3. Awareness that chatbots are capable of learning: knowing that chatbots 
may be trained to learn and polish their skills over time implies that 
the chatbot learns from its queries, behaviours, and actions, which can 
increase (or degrade) its performance. 
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4. Incorporating knowledge from diverse fields: the ability to distinguish 
and recognize the chatbot's generated content, as it may deal with content 
from multiple thematic areas, resulting in inadequate answers for the 
specific subject. 

Ds Understanding concepts: being aware of the context of the created 
information, comprehending it, and comprehending the content that 
the chatbot transmits, while also identifying the chatbot's objective and 
distinguishing between different answer alternatives. 

6. Understanding the logic and programming of chatbots: understanding 
that chatbots are built on certain logic and programming to better grasp 
how the chatbot operates and what is expected of it. 

The analysis of the lesson plans represented an important step as it revealed which 
skills were crucial for the successful writing of lesson plans using the ChatGPT tool. This 
served as a basis for selecting appropriate learning units and learning objectives. 


Data Analysis 


A case study was conducted, focusing on the analysis of 58 lesson plans to examine 
how the use of ChatGPT technology affects their quality and effectiveness. Qualitative 
methods were used for the analysis, with the latter being particularly conducted through 
text analysis techniques. After obtaining all the data, all lesson plans were reviewed and 
analysed. The analysis was conducted by three randomly selected teachers who evaluated 
the lesson plans created with the help of the ChatGPT tool. A short evaluation was written 
from which the key findings were extracted and presented below. In the analysis, teachers 
had to focus mainly on: 

- recording learning objectives and their correctness in relation to the written 

learning objectives in the curriculum, 

- appropriate choice of methods and forms of teaching and learning, 

- is there a list of relevant materials and tools that are key to conducting the 

lesson, 

- timeliness of implementation and 

- general lesson plan. 

Teachers who analysed the lesson plans were required to provide their own opinions 
and feedback on the lesson plans created using the ChatGPT tool. No specific format or 
structure was prescribed for their analyses. Teachers were asked to suggest, among the core 
skills given, which are the essential skills that teachers would need in order to be able to 
successfully prepare lessons using the ChatGPT tool. 


Research Results 
The analysis of the lesson plans revealed that they were partially prepared correctly 
and partially inappropriately (the analysis was based on the comments made about the 


preparations by the instructors who assessed the preparations). The results are presented 
in Table 2. 
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Table 2 
Analysis of the Overview of the Lesson Plans Annotated with ChatGPT tool by Different 
Domains 
Domain Negative Findings Positive Findings 
Most problems were encountered in 
determining the learning objectives, which Some pre-service teachers managed 
were not always properly formulated to translate learning objectives 
according to the learning objectives stated into a learning plan in a very clear 
in the curriculum. Pre-service teachers and understandable way. Learning 
sometimes paid little attention to the objectives were thus set correctly and 
Learning learning objectives, resulting in superficial appropriately. Learning objectives 
Objectives and Sometimes completely inappropriate largely corresponded with the learning 
writing. There were some lesson plans objectives in the curriculum. Teachers 
and Methods : ; eee : 
that did not have any learning objectives who reviewed and analysed the 
written at all, making them completely teaching preparation also pointed 
inappropriate. As a result, inappropriate out that, in cases where the learning 
teaching methods and forms of work objectives were set appropriately, 
were also proposed, which contributed appropriate forms of work were 
to completely inadequate teaching overwhelmingly suggested. 
preparation. 
It was found that the list of necessary In some lesson plans, well-written 
materials for the lessons was often lists of materials and tools for work 
incomplete or inadequate. For example, in were observed and were consistent 
some lesson plans, incorrect or unsuitable with the planned activities. The listed 
tools were listed for a particular activity tools were appropriate, as were the 
Relevant (for example, using Scissors when working materials. 
: with wood). Similarly, during the analysis 
Materials f 
of lesson plans, it was also observed 
that the material list was partially or 
even completely unsuitable. Among the 
materials, there were materials listed that 
were not related to the intended content of 
the lesson. 
The analysis of the lesson plans revealed The scheduling of the lesson plans 
that some of them had only a superficial varies in a similar manner. While 
description of the content and activities, some pre-service teachers adhere to 
making it unclear what the lesson would _ the timeline well, others do not. One 
entail. As a result, the duration of the of the lesson plans was very well 
i lesson was often inappropriate, either too structured in terms of timing and was 
Timeframe 


short to cover the content adequately or too 
long to be completed within a 45-minute 
timeframe. 


planned down to the minute. This 
lesson plan also had a note added that 
it would be good to have an assistant 
present during the lesson to ensure that 
the work is carried out smoothly and 
safely. 


From the analysis of the lessons, the teachers who reviewed and analysed all the 
lesson plans highlighted the following core skills as essential for a teacher to develop a 
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relevant curriculum using ChatGPT in the areas of technology, science and technology 
education: The ability to recognize and troubleshoot issues, Abstract thinking, Awareness 
that chatbots are capable of learning, Incorporating knowledge from diverse fields, 
Understanding concepts and Understanding the logic and programming of chatbots. 


Discussion 


An increasing number of studies (Georgescu, 2018; Hew et al., 2023; Molnar & 
Sziits, 2018; Wollny, 2021) highlight the use of chatbots in education, with their popularity 
on the rise. As stated by Studente et al. (2020), chatbots can have a positive impact on 
both teachers and students. Through our conducted study, it was also found that chatbots, 
particularly ChatGPT, are a useful tool, especially in lesson plan preparation. 

Based on the analysis conducted by the teachers, it was determined that more 
attention needs to be given to the identification of learning objectives and their translation 
into the curriculum. As Chatterjee and Corral (2017) state, the cornerstone of instructional 
alignment lies in the utilization of learning objectives, which facilitate the desired learning 
outcomes by aligning with assessment tools, instructional techniques, and learning 
activities. In some cases, learning objectives were not properly defined, which is likely 
due to inadequate prompts provided to the pre-service teachers in the ChatGPT tool. The 
goal of using the tool was, among other things, to guide pre-service teachers in properly 
communicating with ChatGPT so that the tool can formulate appropriate objectives and 
content. Therefore, it would be advisable to carefully study the learning objectives before 
starting to write the lesson plan and then provide ChatGPT with appropriate prompts. At this 
point, it is believed it is important for the teacher to formulate the learning objectives with 
the help of the curriculum, while ChatGPT can provide suggestions and improvements. 
However, ultimately, the teacher is the one who will make the final decision and determine 
the learning objectives of the lesson. Incorrect or inadequate lists of supplies can also 
present a significant challenge for teachers in preparing and executing lesson plans. With 
inadequate lists, there may be a wrong choice of tools or materials, which can affect the 
quality of the lesson plan and reduce the effectiveness of education. This can also lead to 
wasting time and money on obtaining appropriate materials or correcting mistakes that 
result from the use of inappropriate tools or materials. Lesson planning requires precision 
and attention, especially when it comes to tools and materials lists. If the listed tools and 
materials are correct and appropriate, they can improve the quality of education and the 
execution of the lesson plan. Proper tools and materials are essential for the successful 
implementation of a lesson and help reduce complications or complexity. It is believed 
that at this point, it is also necessary for the teacher to assess whether the listed supply lists 
are correct for the proper implementation of the planned content of the lesson. Certainly, 
the same applies to the content and timing of the lesson plan. With the appropriate sub- 
questions and successful guidance of the ChatGPT tool, the teacher can provide suitable 
prompts that can be a good basis for preparing an appropriate lesson plan. With minor 
adjustments and adaptations of the text generated by ChatGPT, it is possible to prepare an 
appropriate lesson plan. Based on all the above, we can confirm the hypothesis that “With 
the help of the ChatGPT tool, it is possible to write suitable prompts that can be a good 
basis for an appropriate lesson plan”. The analysis of lesson plans created with the help of 
the ChatGPT tool can provide very good prompts that can be translated into an appropriate 
lesson plan with some adjustments and adaptations. 
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The skills listed in the results section represent only the foundational skills, and we 
believe that further research in this area will uncover additional skills necessary. However, 
we acknowledge that there is still much work to be done in this field. As Kooli (2023) also 
suggests, it is essential to raise awareness among users, establish appropriate legislation, 
and adopt ethical guidelines. This can be achieved more effectively once the field is 
thoroughly researched. 


Conclusions and Implications 


The discussion about the analysis of lesson plans prepared using the Chat GPT tool is 
crucial for improving the quality of education and learning, as well as for the use of AI and 
Chat GPT for educational support purposes. As the results of the conducted research show, 
there are some issues that need to be addressed for the lesson plans to be more effective and 
appropriate for the educational goals. Firstly, it would be necessary to focus on improving 
the prompts that users input when preparing their lesson plans using the Chat GPT tool. If 
the prompts were of higher quality, the resulting lesson plans would presumably be more 
appropriate. This is evidenced by those lesson plans that the pre-service teachers upgraded 
multiple times by adding new tasks and instructions to the ChatGPT tool, resulting in more 
precise guidance and better adherence to the content. The appropriate refinement of key 
prompts in lesson plan preparation would lead to more content-appropriate lesson plans. 
Furthermore, more attention needs to be paid to designing learning objectives, as they are 
crucial for achieving appropriate levels of knowledge. A Chat GPT tool user who uses the 
tool for the purpose of preparing lesson plans may understand that learning objectives are 
essential and may need to guide the ChatGPT tool accordingly or independently refine or 
develop the objectives as needed, independent of the ChatGPT tool. The same applies to 
defining tools and materials and the remaining parts of the lesson plan. Before using the 
ChatGPT tool for the purpose of writing lesson plans, it would be necessary to provide 
more support and education to users, so that they understand how to effectively use the 
ChatGPT tool. Since the ChatGPT tool is still relatively new and therefore not fully 
tested, providing education is currently quite challenging. Therefore, it would make sense 
for teachers and other ChatGPT users to come together in a community where they can 
exchange best practices, good examples, and any findings with each other. This would 
create a community that could contribute to the appropriate use of the ChatGPT tool for 
the purpose of writing lesson plans. 
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